Functional assessment of bidirectional cortical and peripheral neural control on heartbeat dynamics: A brain-heart study on thermal stress

The study of functional Brain-Heart Interplay (BHI) from non-invasive recordings has gained much interest in recent years. Previous endeavors aimed at understanding how the two dynamical systems exchange information, providing novel holistic biomarkers and important insights on essential cognitive aspects and neural system functioning. However, the interplay between cardiac sympathovagal and cortical oscillations still has much room for further investigation. In this study, we introduce a new computational framework for a functional BHI assessment, namely the Sympatho-Vagal Synthetic Data Generation Model, combining cortical (electroencephalography, EEG) and peripheral (cardiac sympathovagal) neural dynamics. The causal, bidirectional neural control on heartbeat dynamics was quantified on data gathered from 26 human volunteers undergoing a cold-pressor test. Results show that thermal stress induces heart-to-brain functional interplay sustained by EEG oscillations in the delta and gamma bands, primarily originating from sympathetic activity, whereas brain-to-heart interplay originates over central brain regions through sympathovagal control. The proposed methodology provides a viable computational tool for the functional assessment of the causal interplay between cortical and cardiac neural control

Toward brain-heart computer interfaces: A study on the classification of upper limb movements using multisystem directional estimates

Objective. Brain-computer interfaces (BCIs) exploit computational features from brain signals to perform a given task. Despite recent neurophysiology and clinical findings indicating the crucial role of functional interplay between brain and cardiovascular dynamics in locomotion, heartbeat information remains to be included in common BCI systems. In this study, we exploit the multidimensional features of directional and functional interplay between electroencephalographic and heartbeat spectra to classify upper limb movements into three classes. Approach. We gathered data from 26 healthy volunteers that performed 90 movements; the data were processed using a recently proposed framework for brain-heart interplay (BHI) assessment based on synthetic physiological data generation. Extracted BHI features were employed to classify, through sequential forward selection scheme and k-nearest neighbors algorithm, among resting state and three classes of movements according to the kind of interaction with objects. Main results. The results demonstrated that the proposed brain-heart computer interface (BHCI) system could distinguish between rest and movement classes automatically with an average 90% of accuracy. Significance. Further, this study provides neurophysiology insights indicating the crucial role of functional interplay originating at the cortical level onto the heart in the upper limb neural control. The inclusion of functional BHI insights might substantially improve the neuroscientific knowledge about motor control, and this may lead to advanced BHCI systems performances

Predicting object-mediated gestures from brain activity: an EEG study on gender differences

Recent functional magnetic resonance imaging (fMRI) studies have identified specific neural patterns related to three different categories of movements: intransitive (i.e., meaningful gestures that do not include the use of objects), transitive (i.e., actions involving an object), and tool-mediated (i.e., actions involving a tool to interact with an object). However, fMRI intrinsically limits the exploitation of these results in a real scenario, such as a brain-machine interface (BMI). In this study, we propose a new approach to automatically predict intransitive, transitive, or tool-mediated movements of the upper limb using electroencephalography (EEG) spectra estimated during a motor planning phase. To this end, high-resolution EEG data gathered from 33 healthy subjects were used as input of a three-class k-Nearest Neighbours classifier. Different combinations of EEGderived spatial and frequency information were investigated to find the most accurate feature vector. In addition, we studied gender differences further splitting the dataset into only-male data, and only-female data. A remarkable difference was found between accuracies achieved with male and female data, the latter yielding the best performance (78.55% of accuracy for the prediction of intransitive, transitive and tool-mediated actions). These results potentially suggest that different gender-based models should be employed for future BMI applications

On the Time-Invariance Properties of Upper Limb Synergies

In this paper, we present a novel approach to dynamically describe human upper limb trajectories, addressing the question on whether and to which extent synergistic multi-joint behavior is observed and preserved over time evolution and across subjects. To this goal, we performed experiments to collect human upper limb joint angle trajectories and organized them in a dataset of daily living tasks. We then characterized the upper limb poses at each time frame through a technique that we named repeated-principal component analysis (R-PCA). We found that, although there is no strong evidence on the predominance of one principal component (PC) over the others, the subspace identified by the first three PCs takes into account most of the motion variability. We evaluated the stability of these results over time, showing that during the reaching phase, there is a strong consistency of these findings across participants. In other words, our results suggest that there is a time-invariant low-dimensional approximation of upper limb kinematics, which can be used to define a suitable reduced dimensionality control space for upper limb robotic devices in motion phases

Improving Emotion Recognition Systems by Exploiting the Spatial Information of EEG Sensors

Electroencephalography (EEG)-based emotion recognition is gaining increasing importance due to its potential applications in various scientific fields, ranging from psychophysiology to neuromarketing. A number of approaches have been proposed that use machine learning (ML) technology to achieve high recognition performance, which relies on engineering features from brain activity dynamics. Since ML performance can be improved by utilizing 2D feature representation that exploits the spatial relationships among the features, here we propose a novel input representation that involves re-arranging EEG features as an image that reflects the top view of the subject’s scalp. This approach enables emotion recognition through image-based ML methods such as pre-trained deep neural networks or "trained-from-scratch" convolutional neural networks. We have employed both of these techniques in our study to demonstrate the effectiveness of our proposed input representation. We also compare the recognition performance of these methods against state-of-the-art tabular data analysis approaches, which do not utilize the spatial relationships between the sensors. We test our proposed approach using two publicly available benchmark datasets for EEG-based emotion recognition tasks, namely DEAP and MAHNOB-HCI. Our results show that the "trained-from-scratch" convolutional neural network outperforms the best approaches in the literature, achieving 97.8% and 98.3% accuracy in valence and arousal classification on MAHNOB-HCI, and 91% and 90.4% on DEAP, respectively

From local counterfactuals to global feature importance: efficient, robust, and model-agnostic explanations for brain connectivity networks

Background: Explainable artificial intelligence (XAI) is a technology that can enhance trust in mental state classifications by providing explanations for the reasoning behind artificial intelligence (AI) models outputs, especially for high-dimensional and highly-correlated brain signals. Feature importance and counterfactual explanations are two common approaches to generate these explanations, but both have drawbacks. While feature importance methods, such as shapley additive explanations (SHAP), can be computationally expensive and sensitive to feature correlation, counterfactual explanations only explain a single outcome instead of the entire model. Methods: To overcome these limitations, we propose a new procedure for computing global feature importance that involves aggregating local counterfactual explanations. This approach is specifically tailored to fMRI signals and is based on the hypothesis that instances close to the decision boundary and their counterfactuals mainly differ in the features identified as most important for the downstream classification task. We refer to this proposed feature importance measure as Boundary Crossing Solo Ratio (BoCSoR), since it quantifies the frequency with which a change in each feature in isolation leads to a change in classification outcome, i.e., the crossing of the model's decision boundary. Results and conclusions: Experimental results on synthetic data and real publicly available fMRI data from the Human Connect project show that the proposed BoCSoR measure is more robust to feature correlation and less computationally expensive than state-of-the-art methods. Additionally, it is equally effective in providing an explanation for the behavior of any AI model for brain signals. These properties are crucial for medical decision support systems, where many different features are often extracted from the same physiological measures and a gold standard is absent. Consequently, computing feature importance may become computationally expensive, and there may be a high probability of mutual correlation among features, leading to unreliable results from state-of-the-art XAI methods

Fine-Grained Emotion Recognition Using Brain-Heart Interplay Measurements and eXplainable Convolutional Neural Networks

Emotion recognition from electro-physiological signals is an important research topic in multiple scientific domains. While a multimodal input may lead to additional information that increases emotion recognition performance, an optimal processing pipeline for such a vectorial input is yet undefined. Moreover, the algorithm performance often compromises between the ability to generalize over an emotional dimension and the explainability associated with its recognition accuracy. This study proposes a novel explainable artificial intelligence architecture for a 9-level valence recognition from electroencephalographic (EEG) and electrocardiographic (ECG) signals. Synchronous EEG-ECG information are combined to derive vectorial brain-heart interplay features, which are rearranged in a sparse matrix (image) and then classified through an explainable convolutional neural network. The proposed architecture is tested on the publicly available MAHNOB dataset also against the use of vectorial EEG input. Results, also expressed in terms of confusion matrices, outperform the current state of the art, especially in terms of recognition accuracy. In conclusion, we demonstrate the effectiveness of the proposed approach embedding multimodal brain-heart dynamics in an explainable fashion

Heart rate variability in marketing research: A systematic review and methodological perspectives

Heart rate variability is a promising physiological measurement that accesses psychophysiological variations in response to a marketing stimulus. While its application spans diverse fields, there is a limited understanding of the usability and interpretation of heart rate variability in marketing research. Therefore, this hybrid literature review provides an overview of the emerging use of heart rate variability in marketing research, along with essential methodological considerations. In this context, we blend marketing mix framework with stimulus-organism-response theory, segregating the use of heart rate variability in various marketing research contexts. We follow the preferred reporting items for systematic reviews and meta-analyses (PRISMA) framework to reflect on 33 records obtained from six databases. Our findings suggest that 42% of studies used heart rate variability to investigate promotion-related topics. Overall, heart rate variability is mostly used in combination with Galvanic skin response (48%). Further, 39% of studies used non-portable systems for data collection. Last, using the theory characteristics methodology (TCM) framework, we identified six research avenues: (1) affective, cognitive, and sensorial constructs; (2) personality, thinking style, and demographics; (3) product experience; (4) advertising and branding; (5) correlation with immersive technologies; and (6) triangulation with other neurophysiological tools

U-Limb: A multi-modal, multi-center database on arm motion control in healthy and post-stroke conditions

BACKGROUND: Shedding light on the neuroscientific mechanisms of human upper limb motor control, in both healthy and disease conditions (e.g., after a stroke), can help to devise effective tools for a quantitative evaluation of the impaired conditions, and to properly inform the rehabilitative process. Furthermore, the design and control of mechatronic devices can also benefit from such neuroscientific outcomes, with important implications for assistive and rehabilitation robotics and advanced human-machine interaction. To reach these goals, we believe that an exhaustive data collection on human behavior is a mandatory step. For this reason, we release U-Limb, a large, multi-modal, multi-center data collection on human upper limb movements, with the aim of fostering trans-disciplinary cross-fertilization. CONTRIBUTION: This collection of signals consists of data from 91 able-bodied and 65 post-stroke participants and is organized at 3 levels: (i) upper limb daily living activities, during which kinematic and physiological signals (electromyography, electro-encephalography, and electrocardiography) were recorded; (ii) force-kinematic behavior during precise manipulation tasks with a haptic device; and (iii) brain activity during hand control using functional magnetic resonance imaging

Computational modeling and analysis of ECoG data through Support Vector Machines algorithms.

In this work some ElectoCorticography data has been taken in account, a multivariate analysis has been proposed considering spatial and time-frequency features. The Dataset has been recorded following the Recalibration and Adaptation paradigm about ambigous and not-ambigous audio-visual stimulation. A classification between these two conditions through a Support Vector Machine algorithm using a RBF kernel has been implemented. An intra- and inter- subject discussion of the results will close the dissertation